1,686 research outputs found

    STARC: Structured Annotations for Reading Comprehension

    Full text link
    We present STARC (Structured Annotations for Reading Comprehension), a new annotation framework for assessing reading comprehension with multiple choice questions. Our framework introduces a principled structure for the answer choices and ties them to textual span annotations. The framework is implemented in OneStopQA, a new high-quality dataset for evaluation and analysis of reading comprehension in English. We use this dataset to demonstrate that STARC can be leveraged for a key new application for the development of SAT-like reading comprehension materials: automatic annotation quality probing via span ablation experiments. We further show that it enables in-depth analyses and comparisons between machine and human reading comprehension behavior, including error distributions and guessing ability. Our experiments also reveal that the standard multiple choice dataset in NLP, RACE, is limited in its ability to measure reading comprehension. 47% of its questions can be guessed by machines without accessing the passage, and 18% are unanimously judged by humans as not having a unique correct answer. OneStopQA provides an alternative test set for reading comprehension which alleviates these shortcomings and has a substantially higher human ceiling performance.Comment: ACL 2020. OneStopQA dataset, STARC guidelines and human experiments data are available at https://github.com/berzak/onestop-q

    Changing practice in Malaysian primary schools: learning from student teachers’ reports of using action, reflection and modelling (ARM)

    Get PDF
    This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of Education for Teaching on 15 March 2018, available online at doi: https://doi.org/10.1080/02607476.2018.1433468. Under embargo until 1 August 2019.Curricular and pedagogical reforms are complex inter-linked processes such that curricular reform can only be enacted through teachers teaching differently. This article reports the perspective of emergent Malaysian primary teachers who were expected to implement a Government reform that promoted active learning. The 120 student teachers were members of a single cohort completing a new B.Ed. degree programme in Primary Mathematics designed by teacher educators from Malaysia and the UK. They were taught to use a tripartite pedagogical framework involving action or active learning, supported in practice through reflection and modelling. Drawing on findings from surveys carried out with the student teachers at the end of their first and final placements this article examines evidence for the premise that the student teachers were teaching differently; illustrates how they reported using active learning strategies; and identifies factors that enabled and constrained pedagogic change in the primary classroom. The students’ accounts of using action, reflection and modelling are critiqued in order to learn about changing learning and teaching practice and to contribute to understanding teacher education and early teacher development. The students’ reports suggest diversity of understanding that emphasises the need to challenge assumptions when working internationally and within national and local cultures.Peer reviewe
    • …
    corecore